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What GAO Found 

The Department of Education’s (Education) Race to the Top (RTT) program 
encouraged states to reform their K-12 educational systems, but states and 
districts faced various capacity challenges in implementing the reforms. RTT 
accelerated education reforms underway and spurred new reforms in all 19 RTT 
states and in an estimated 81 percent of districts, according to GAO’s surveys of 
RTT grantees and districts that received RTT funds. At the same time, states and 
districts noted various challenges to their capacity to successfully support, 
oversee, and implement these reform efforts. For example, about one-quarter to 
one-third of RTT states reported that their greatest challenges involved obtaining 
support from stakeholders such as teacher organizations. In contrast, districts 
primarily reported that their greatest challenges involved financial and human 
capital capacity, especially with competitive compensation and standards and 
assessments. Additionally, rural districts reported facing greater challenges than 
urban and suburban districts. Education is to assist grantees in achieving 
successful project outcomes according to its grants handbook, while holding 
them accountable for their RTT reform plans. Yet, GAO found no specific 
activities tailored to rural needs in areas grantees identified as most challenging. 
A better understanding of the capacity challenges rural districts face could help 
Education better target its technical assistance to districts that need it the most. 

In response to GAO’s survey, many RTT states reported that technical 
assistance from Education officials and its contractor was more helpful than other 
RTT resources, such as web-based materials. Ten states also reported they 
would benefit from additional support in areas such as training and professional 
development. Education created a new office to oversee and provide coordinated 
support to RTT and other programs, and intends to develop office-wide 
coordinated technical assistance policies. Federal internal control standards note 
that adequate policies help ensure that actions are taken to address risks to 
achieving an agency’s objectives. However, Education has not determined the 
type or amount of technical assistance to be provided and its policies are still 
being developed. RTT’s $43 million technical assistance contract ends in June 
2015, which may create a gap in assistance to states. Unless Education focuses 
on technical assistance activities that states found most useful, it risks providing 
ineffective assistance to programs supporting these education reforms. 

GAO’s panel of RTT and grant experts identified key lessons learned, such as 
leveraging existing funding flexibilities under federal formula grants, to help 
address capacity needs and sustain reforms when RTT ends in September 2015. 
Districts and schools may not, however, be using these flexibilities to their fullest 
extent, in part because of uncertainty about what is allowed under federal 
requirements. Federal internal control standards state that information should be 
communicated in a form that enables an agency to achieve its objectives. 
Education lacks time frames for finalizing and disseminating new guidance for 
states to clarify federal formula grant flexibilities; and recognizes the need for, but 
has not developed guidance to help auditors better understand these flexibilities. 
Such guidance, when finalized, may help states and districts sustain education 
reforms, thereby raising student achievement - a primary objective of reform. 
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441 G St. N.W. 
Washington, DC 20548 


U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


April 13, 2015 

The Honorable Rosa L. DeLauro 
Ranking Member 

Subcommittee on Labor, Health and Human Services, Education and 
Related Agencies 
Committee on Appropriations 
House of Representatives 

Dear Congresswoman DeLauro: 

From 2010 through 2011, the Department of Education (Education) 
awarded over $4 billion in Race to the Top (RTT) grant funds to 19 states. 
These RTT grantees serve about 22 million students in more than 40,000 
schools, representing 45 percent of all K-12 students and 42 percent of all 
low-income students nationwide.'' Through a competitive grant process — 
the largest ever administered by Education — these states had to 
demonstrate a commitment to reforming four core areas of K-12 
education: (1) standards and assessments; (2) data systems; (3) effective 
teachers and leaders; and (4) school turnaround. States also committed 
to build strong statewide capacity to implement, scale up, and sustain 
their plans. Many RTT states are also implementing similar reform efforts 
as a condition of obtaining Elementary and Secondary Education Act of 
1965 (ESEA) waivers or funds through other Education programs, such 
as School Improvement Grants.^ At the same time, states and local 
governments continue to face funding gaps that threaten to adversely 
affect educational services. 

GAO and others have reported that state and district capacity challenges 
have adversely affected the implementation of RTT. For example, in 
201 1 , we reported that most RTT states faced a variety of challenges that 
led to implementation delays, including difficulty hiring qualified 


^ The White House and U.S. Department of Education, Setting the Pace: Expanding 
Opportunity for America’s Students under Race to the Top. (Washington, D.C.: March 
2014),- see also http://www.whitehouse.gov/issues/education/k-12/race-to-the-top. 

^ Education is authorized to waive ESEA requirements in certain circumstances. States 
use School Improvement Grants to make competitive sub-grants to districts that 
demonstrate need and commitment to providing resources to low-performing schools. 
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personnel.^ In September 2013, we found that most RTT states did not 
meet their target date for implementing teacher and principal evaluation 
reforms, citing capacity issues, such as staff expertise, among their 
challenges. Education officials from most of the 12 states and 12 districts 
we interviewed for our 2013 report also cited concerns about their ability 
to sustain evaluation systems after RTT grant funds are no longer 
available. You asked us to examine broader capacity challenges states 
and districts face as they implement all RTT reform efforts. 

In this report, we examine: (1) the effect RTT had on education reform, 
and the capacity challenges states and districts faced in implementing 
and sustaining RTT reform efforts; (2) states’ perspectives on how helpful 
Education’s assistance was in helping them build and sustain capacity to 
implement RTT reforms; and (3) lessons learned from RTT that could 
inform future education reform efforts.® 

To examine capacity challenges at the state and district level we 
conducted two surveys. We sent one survey to state educational agency 
officials in all 19 RTT grantee states, including the District of Columbia.® 
We sent a second survey to a stratified random sample of 643 school 
districts that received RTT funds — the results of which were generalizable 
to all districts receiving RTT funds. We administered our web-based 
surveys from May through September 2014. All state officials and 76.7 
percent of district officials in our sample responded to the surveys.^ The 


® GAO, Race to the Top: Characteristics of Grantees’ Amended Programs and 
Education’s Review Process, GAO-1 2-228R (Washington, D.C.: December 8, 201 1 ) and 
GAO, Race to the Top: Reform Efforts Are Under Way and information Sharing Couid Be 
improved, GAO-11-658 (Washington, D.C.: June 30, 2011). 

^ GAO, Race to the Top: States tmpiementing Teacher and Principai Evaiuation Systems 
despite Chaiienges, GAO-13-777 (Washington, D.C.: September 18, 2013). The 
methodology for this study included interviews with officials from 12 RTT states and 12 
RTT districts within those states. 

® Education has also awarded other RTT grants since the inception of RTT, including 
those under the Assessment, Early Learning Challenge, and District programs; however, 
these RTT programs are outside the scope of this study. 

® In this report, we refer to the District of Columbia as a state. 

^ Estimates based on our survey of district officials are based on a random sample and 
subject to sampling error. All percentage estimates used in this report have 95 percent 
confidence intervals and margins of error of within +/- 6 percentage points, unless 
otherwise noted. 
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surveys and a more complete tabulation of the results can be viewed at 
GAO-1 5-31 6SP and GAO-1 5-31 7SP. We also conducted follow-up 
interviews with officials from four state educational agencies and three 
districts that received RTT funds to obtain specific examples of their 
challenges, support received, and efforts to build and sustain their 
capacity for reform.® In addition, we reviewed documentation, including 
states’ RTT applications and annual performance reports to understand 
their capacity challenges and efforts to build capacity before receiving 
RTT funds and throughout the grant period. To understand the assistance 
Education provided to states, we met with Education’s RTT program 
officials to discuss their efforts, particularly related to capacity, monitoring, 
oversight, and technical assistance. We also gathered information on 
Education’s assistance from our survey and in our follow-up interviews 
with selected state officials. 

To ascertain how lessons learned from RTT could inform future education 
reform efforts, we held an expert panel that addressed the implications of 
state and district capacity issues on sustaining RTT reforms and future 
competitive grants. We developed discussion topics and questions for the 
panelists based on information gathered from our surveys, interviews, 
and academic literature. We selected experts with research or 
professional experience related to RTT and other competitive grants, 
state and district capacity, and federal grant making. See appendix V for 
a list of panelists and appendix I for more details on our scope and 
methodology, including specifics on our survey and expert panel. 

We conducted this performance audit from November 201 3 to April 201 5 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


® We selected the states and districts for follow-up interviews based on their responses to 
our surveys and representation across award phase. 
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Background 


0v6rvi6W of RTT The American Recovery and Reinvestment Act of 2009 (Recovery Act) 

required the Secretary of Education to provide grants to states that show 
promise in meeting the objectives of four broad education reform areas 
outlined in law.® Education subsequently established the RTT grant fund 
to encourage states to reform their K-12 education systems and to reward 
states for improving certain student outcomes, such as making 
substantial gains in student achievement and improving high school 
graduation rates. The reforms contained in RTT were expected to help 
prepare students to graduate ready for college and career, and enable 
them to successfully compete with workers in other countries. Providing a 
high-quality education for every student is also vital to a strong U.S. 
economy. 

States competed for RTT grant funds based on reforms across the 
following four core reform areas: 

1. Standards and assessments; adopting standards and assessments 
that prepare students to succeed in college and the workplace and to 
compete in the global market; 

2. Data systems: building data systems that measure student academic 
growth and success and inform teachers and principals about how 
they can improve instruction; 

3. Effective teachers and leaders: recruiting, developing, rewarding, 
and retaining effective teachers and principals, especially where they 
are needed most; and 

4. School turnaround: turning around the lowest-achieving schools.^® 


® Pub. L No. 111-5, §§ 14005-06, 123 Stat. 115, 283. 

1 n 

States are encouraged to use one of four school intervention models to turn around 
low-performing schools — turnaround, restart, school closure, or transformation. Under 
these models, states may need to take actions such as replacing principals and staff, 
reopening schools under different management, or closing schools. 
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Education awarded RTT grants to states in three phases, with award 
amounts ranging from approximately $17 million to $700 million (see 
appendix II for list of grantees and award amounts). States are generally 
required to sub-grant at least 50 percent of their RTT funds to school 
districts within their state that signed a Memorandum of Understanding 
stating their agreement to implement all or significant portions of the 
state’s RTT plan (participating districts).'’'' According to Education 
officials, providing a competitive grant with substantial funding to 
implement ambitious plans in the four core education reform areas was 
meant to encourage states to create the conditions for reform and 
achieve significant improvement in student outcomes (see fig. 1). The 4- 
year grant period began on the date funds were awarded to the state. 
Education officials stated that, of the Recovery Act funding used in 2010 
for the first two phases of RTT, under federal law, any funds not obligated 
and liquidated by September 30, 2015, will no longer be available.'’^ 
Education made grants for the third phase of RTT from fiscal year 201 1 
funding, and officials told us that those funds must be liquidated by 
September 30, 2017. 


In this report, we use the term school districts or districts to refer to local educational 
agencies. 

States are generally required to obligate all funds within the grant period. States have 
90 days following the end of their grant period to liquidate all obligated funds unless they 
receive a no-cost extension. Education officials stated that they have advised states to 
stop obligating funds by June 30, 2015, which leaves them 3 months remaining for the 
normal 90-day liquidation period. 
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Figure 1 : Race to the Top (RTT) Overview and Award Process 
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In awarding the RTT grants, Education used a peer review process to 
evaluate applications. Capacity to implement, scale up, and sustain RTT 
reforms was one of 19 primary criteria Education used to guide the 
selection of RTT grantees (see appendix III for a list of these criteria).''^ 
Education did not provide a definition of capacity, but it provided guidance 
to peer reviewers on how to assess the specific criterion related to 
capacity: building strong statewide capacity to implement, scale up, and 
sustain proposed plans. Peer reviewers evaluated states on the extent to 
which they demonstrated that they would: (1) provide strong leadership 
and dedicated teams to implement the reforms; (2) support participating 
districts in implementing the reforms through a variety of activities, such 
as identifying and disseminating promising practices; (3) provide efficient 
and effective operations and processes for grant administration and 
performance measurement, among other functions; (4) use RTT funds to 
accomplish the state’s plans; and (5) use fiscal, political, and human 
capital resources to continue successful grant-funded reforms after RTT 
funds are no longer available.'''^ 


Capacity to Implomont The capacity of grantees is a key issue in grants management that can 

RTT affect program success. Capacity involves both maintaining appropriate 

resources and the ability to effectively manage those resources. For the 
purposes of this report, we defined capacity as the ability to successfully 
support, oversee, and implement reform efforts.''^ It includes the following 
types of capacity: 

• Organizational Capacity: degree of preparedness for grants 
management and implementation including having the appropriate 


Race to the Top Fund, 74 Fed. Reg. 59,688 (Nov. 18, 2009). At Education’s invitation, 
over 1 ,500 prospective reviewers applied or were nominated to review Phase 1 RTT 
applications. Education ultimately selected 58 reviewers. 

See 74 Fed. Reg. 59,689, 59,801 (Nov. 18, 2009), criterion (A)(2) for Education’s 
criteria related to state capacity. 

We developed this definition based on prior work on capacity-related issues conducted 
by GAO and other researchers. See GAO, Grants to State and Local Governments: An 
Overview of Federal Funding Levels and Selected Challenges, GAO-1 2-1 01 6 
(Washington, D.C.: September 25, 2012), pp. 27-29; and Jochim, Ashley and Murphy, 
Patrick, The Capacity Challenge: What It Takes for State Education Agencies to Support 
School Improvement, Center on Reinventing Public Education (Seattle, WA: December 
2013). 
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leadership, management, and structure to efficiently and effectively 
implement the program and adapt as needed. 

• Human Capital Capacity: the extent to which an organization has 
sufficient staff, knowledge, and technical skills to effectively meet its 
program goals. 

• Financial Capacity: the extent to which an organization has sufficient 
financial resources to administer or implement the grant. 

• Stakeholder Capacity: the extent to which an organization has 
sufficient support from its stakeholders, including their authority and 
commitment to execute reform efforts.''® 

We and other researchers have noted that capacity concerns may have 
important implications for competitive grants generally. For example, in 
201 1 and 2012, we reported on the School Improvement Grant program, 
another competitive grant awarded by Education, and found that human 
capital and stakeholder capacity issues influenced the implementation of 
School Improvement Grant interventions.''^ In addition, a 2011 Journal of 
Federalism study demonstrated that applicant capacity is an important 
factor likely to influence how competitive grants are administered and that 
an applicant’s chances of winning competitive grants are strongly related 
to their capacity.''® Other researchers also raised concerns about states’ 
capacity given relatively modest levels of investment in school 
improvement activities, as well as human resources, organization, and 
political challenges. In a January 2014 report. Education’s Inspector 
General identified common capacity-related causes for delays, such as 
changes in state leadership; staffing and organizational challenges at 


Stakeholders could include state leadership (e.g., Governor, state legislature), district 
leadership (e.g.. Superintendent), and organizations that represent teachers and 
administrators, among others. 

GAO, School Improvement Grants: Early Implementation Under Way, but Reforms 
Affected by Short Time Frames, GAO-11-741 (Washington, D.C.: July 25, 201 1 ). See also 
GAO-12-373. 

Manna, Paul and Ryan, Laura L., Competitive Grants and Educational Federalism: 
President Obama’s Race to the Top Program in Theory and Practice, Publius: The Journal 
of Federalism, vol. 41 , no. 3, pp. 522-546, 201 1 . 
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state educational agencies; acquisitions issues; and stakeholder issues, 
particularly regarding the new evaluation systems.''® 


Rol© of thl6 Dopartmont of in 2011, Education established the Implementation and Support Unit, 
Education within the Office of the Deputy Secretary, to administer the RTT program. 

The purpose of the Implementation and Support Unit was to support the 
implementation of comprehensive reforms at the state level, pilot new 
approaches to strengthen and support state reforms, and act as a single 
point of contact for the Education programs that were housed in that 
office.^® The office was responsible for fiscal and programmatic oversight 
of all aspects of RTT, including monitoring and technical assistance. 

The Implementation and Support Unit established a program review 
process to monitor RTT states’ progress toward meeting their RTT goals 
and to tailor support based on individual state needs. The program review 
process emphasized outcomes and the quality of RTT implementation by 
states rather than focusing solely on a compliance-driven approach. 
Program officials and other staff in the Implementation and Support Unit 
were to work directly with states to understand their RTT plans and 
objectives, observe benchmarks, and monitor the quality of 
implementation. Education considered each state’s progress toward its 
goals and timelines, risk factors and strategies for addressing them, and 
the state’s own assessment of its quality of implementation, among other 
factors. In October 2014, Education established a new Office of State 
Support, which replaced the Implementation and Support Unit in the 
administration and oversight of RTT. 

Education provides technical assistance to RTT states via the Reform 
Support Network (RSN), which it established in 2010 through a 4-year, 
$43 million technical assistance contract with ICF International.^' The 


1 Q 

U.S. Department of Education Office of Inspector General, The Department’s 
Monitoring of Race to the Top Program Recipient Performance, ED-OIG/A19M0003 
(Washington, D.C.: January 3, 2014). 

In addition to RTT, the Implementation and Support Unit was responsible for overseeing 
the RTT Assessment, Education Jobs Fund, and the State Fiscal Stabilization Fund 
programs. 

ICF International is a consulting firm that provides professional services and technology 
solutions to government and commercial clients. 
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RSN is intended to work with RTT states to build capacity to implement 
and sustain reform efforts and achieve improvements in educational 
outcomes, identify and share promising and effective practices, as well as 
facilitate collaboration across states and among the many education 
stakeholders who implement and support state reform efforts. RSN is to 
provide RTT grantees one-on-one technical assistance that is tailored to 
the grantee’s RTT reform plans. RSN is to ensure that the state 
requesting individualized technical assistance receives the best available 
and relevant expertise by identifying specific experts that a state can 
contact for help. 

RSN also provides collective technical assistance to RTT states through 
communities of practice. Communities of practice use a variety of 
mechanisms to support states in meeting their RTT goals, including the 
use of working groups, publications, and various forms of direct technical 
assistance, such as webinars and individualized technical assistance. 
RSN established a capacity-building community of practice designed to 
strengthen the organizational capacity of RTT states and a working group 
to help states assess the sustainability of their reform initiatives and take 
action if needed. 


RTT Spurred Reform 
amid Different Types 
of State and District 
Capacity Challenges 


RTT Both Accelerated 
Reforms Already Under 
Way and Spurred New 
Reform Efforts 


RTT accelerated reforms underway or spurred new reforms in all 19 
states and in an estimated 81 percent of districts that were awarded RTT 
grants, according to states and districts we surveyed (see fig. 2 for district 
survey responses). For example, several state officials reported in their 
survey comments that their states began implementing reform activities — 
such as developing standards, longitudinal data systems, and new 


Specifically, RSN established a community of practice on each of the following areas: 
(1) teacher and leader effectiveness/standards and assessment, (2) school turnaround, 
(3) stakeholder communications and engagement, (4) instructional improvement/data 
systems and (5) capacity-building for states and districts. 
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teacher evaluation systems — before they received RTT funds. In addition, 
16 states reported that RTT provided the opportunity to accelerate or 
enhance existing reform plans or existing priorities. For example, one 
state official reported that RTT allowed their state to increase courses in 
science, technology, engineering, and math for students and teachers 
and provide professional development opportunities for pre-kindergarten 
teachers. 


Figure 2: Overall Effect of Race to the Top (RTT) on Reform Efforts, as Estimated 
for Districts 



Both accelerated reform plans already 
underway and allowed for development 
of new reform plans 

A negative effect on reform plans 

Don’t know 


No effect on reform plans 
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Source: GAO survey of RTT participating districts. | GAO-15-295 

Note: This figure inciudes data from aii districts participating in RTT, inciuding those that were initiaiiy 
participating but iater withdrew. Estimates have 95 percent confidence intervais of within +/- 6 
percentage points of the estimate itseif. 


In addition, RTT may have helped promote reforms not only within the 19 
states that received RTT grants, but also in the states that applied but did 
not receive RTT funding. A 2014 Education study found that although 
RTT states implemented more reform activities in the four core reform 
areas than non-RTT states, many non-RTT states also adopted similar 
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reforms. Specifically, many of the 47 states that applied for the grant 
had aligned their educational policies and actions to RTT’s four core 
education reform areas to develop a competitive application. For 
example, 43 states had adopted Common Core State Standards 
(Common Core) in both math and reading/English language arts in the 
2010-1 1 school year.^"^ Adopting college- and career-ready standards was 
one of the 19 criteria peer reviewers used to select RTT grantees. 
Similarly, our prior work on RTT found that four states that applied for but 
were not awarded a RTT grant reported enacting new state legislation or 
making formal executive branch policy changes to be more competitive 
for RTT. Further, our 201 1 report found that sharing information with all 
states carrying out initiatives similar to RTT initiatives can accelerate the 
pace and scope of reform efforts. Education developed RTT resources 
and subsequently made them available to all states on its website.^® 


States and Districts Faced 
Moderate but Different 
Capacity Challenges 
Implementing RTT 


In our survey of states and districts that received RTT funds, we asked 
officials to identify capacity challenges they faced in implementing and 
sustaining RTT and the level of difficulty associated with each challenge 
identified. In general, capacity issues posed a somewhat moderate level 
of challenge to states and currently participating districts implementing 
RTT, according to our survey of states and districts that received RTT 
funds. However, some states and districts described particular aspects 


The researchers developed a set of 18 indicators of state-level reform that reflected 
Education’s priorities and key reform strategies within each of the four core reform areas. 
Webber, A., Troppe, P., Milanowski, A., Gutmann, B., Reisner, E., and Goertz, M. State 
Implementation of Reforms Promoted Underthe Recovery Act {NCEE 2014-4011). 
Washington, D.C.: National Center for Education Evaluation and Regional Assistance, 
Institute of Education Sciences, U.S. Department of Education (Washington, D.C., 2014). 

The Council of Chief State School Officers and the National Governors Association 
brought together states to develop common college- and career-ready standards for 
grades K through 12 in math and English, which resulted in common math and English 
standards known as the Common Core State Standards which were published in 2010. 

GAO-1 1-658. 

http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/index.html. 

For the purpose of describing survey results, currently participating districts are defined 
as districts that were receiving RTT grant funds at the time of our survey. It excludes 
districts that never participated in RTT as well as districts that had previously participated 
in RTT, but later formally withdrew from the program. 
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of the four types of capacity — organizational, human capital, financial and 
stakeholder — as very or extremely challenging. 

For example, RTT states rated stakeholder capacity as the greatest 
challenge faced while implementing RTT reform initiatives. Overall, they 
rated this challenge as moderate; however, about one-quarter to one-third 
of RTT states reported that obtaining support from state legislatures, 
organizations that represent teachers and/or administrators, and district 
leaders was very or extremely challenging.^® Further, in implementing 
changes in two of the four core reform areas — standards and 
assessments and effective teachers and leaders — more than one-third of 
RTT states found stakeholder capacity to be very or extremely 
challenging.^® Although states were encouraged to show in their grant 
applications that they had garnered support for reforms from 
stakeholders, some states said that they had difficulty maintaining that 
support throughout the grant period. One state official told us that the 
state’s teachers’ union was seeking to reverse elements of their 
evaluation system linking teacher performance to student achievement, 
and the legislature was seeking to reverse the adoption of the Common 
Core — key elements of the state’s RTT application. 

RTT states rated organizational capacity as the second greatest 
challenge faced while implementing RTT. Although they rated this 
challenge as moderate overall, officials from 4 of the 19 states reported 
that consistency in leadership at the state educational agency was a 
specific aspect of organizational capacity that was very or extremely 
challenging.®® Cne state official we spoke with explained that frequent 
turnover at the superintendent level made implementing its teacher 
evaluation system difficult because they had to constantly educate new 
superintendents on how to use the evaluations to improve instruction. 

School districts reported facing different types of capacity challenges than 
did states. For example, school districts currently participating in RTT 


oo 

stakeholder capacity is the extent to which states had sufficient support from their 
stakeholders, including their authority and commitment to execute reform efforts. 

See appendix IV for more information on capacity challenges by reform area. 

OA 

Organizational capacity is the extent to which an entity is prepared to manage and 
implement grants, including having the appropriate leadership, management, and 
structure to efficiently and effectively implement a program and adapt as needed. 
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reforms reported that financial capacity was the most challenging. In each 
of the four core reform areas, about one-third of currently participating 
districts reported that financial capacity was very or extremely challenging 
to implementing RTT initiatives (see appendix IV). District officials we 
surveyed stated in their written comments that decreased state funding, 
the effects of the 2008 recession, and increasing enrollments affected 
their financial capacity to fund reform at the local level. While RTT grant 
funding to currently participating districts represented an estimated 1 to 2 
percent of their budgets during each school year of the grant period, 
district officials told us that RTT funds were crucial to their ability to 
implement reforms. 

Districts also reported particular difficulties with human capital capacity — 
the second greatest challenge they faced implementing RTT.^^ Districts 
currently participating in RTT reported the most challenging aspect of 
human capital capacity was recruiting staff through competitive 
compensation, with an estimated 45 percent of districts reporting that 
doing so was very or extremely challenging. An estimated one-third of 
currently participating districts also cited retaining staff and having the 
appropriate number of staff among the most challenging aspects of 
human capital capacity, as well as issues related to Common Core 
implementation, such as having staff prepared to develop and/or 
implement curricula meeting the new standards. 


These estimates and their corresponding confidence intervals are 1.68 (+/-1.19), 1.21 
(+/- .53), .91 (+/- .32), and .88 (+Z-.27) for 201 1 through 2014, respectively. 

oo 

This amount refers only to RTT grant funds, and does not include funds from RTT- 
district grants, which are separate grants made directly to districts. 

Human capital capacity is the extent to which an organization has sufficient staff, 
knowledge, and technical skills to effectively meet its program goals. 

For more information about challenges states face in implementing new college and 
career ready standards, see GAO, College- and Career-Readiness: States Have Made 
Progress In Implementing New Standards and Assessments, but Challenges Remain, 
GAO-15-104R (Washington, D.C.: December 12, 2014). 
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Most States and Districts 
Increased Capacity During 
the Grant Period but 
Anticipate Challenges 
Sustaining Their Efforts 


States and districts reported taking various actions to build and increase 
their capacity overall throughout the grant period (see fig. 3). However, 
both indicated that human capital and financial capacity would be the 
most challenging to sustain after the RTT grant period ends. State and 
district officials we spoke with explained that these issues were inter- 
related; that is, staff shortages and skill gaps required continued funds for 
professional development. 


Figure 3: Increase in Overall Capacity to Implement Race to the Top (RTT) Reforms from Time of Award to Present, As 
Reported by States and Estimated by Districts 


State survey data 


District survey data 


At the time of 
your RTT award 


No capacity/ Moderate capacity to 
minimal capacity < ^ almost full/full capacity 

H I 


Currently 



No capacity/ Moderate capacity to 
minimal capacity -4 ^ almost full/full capacity 




No capacity/minimal capacity 

Moderate capacity 

Aimost fuii capacity/fuii capacity 


Source: GAO survey of RTT states and currentiy participating districts. | GAO-15-295 


Note: The district estimates have 95 percent confidence intervals of within +/- 6 percentage points of 
the estimates themselves. 


Throughout the grant period, more than half of the 19 states reported 
putting great or very great effort into building stakeholder capacity — the 
area that state officials cited as the most challenging — most frequently by 
consulting with organizations that represent teachers and/or 
administrators (17 states), consulting with district leadership (16 states), 
and building political relationships (15 states). 

Similarly, most states reported building organizational capacity — another 
area that presented great challenges as they implemented reforms — by, 
for example, establishing an RTT point of contact or office (18 states) and 
establishing communication mechanisms for RTT staff, such as group 
email lists (17 states). To a lesser extent, states reported that 
reorganizing an existing office (12 states) and appointing new RTT 
leadership (13 states) were also helpful in building organizational 
capacity. According to one state official we spoke with, the state 
reorganized its entire state educational agency into departments aligned 
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with its RTT reforms. The official noted that the RTT grant helped the 
state fund the reorganization which, in turn, helped them mitigate capacity 
challenges throughout implementation. Another state official explained 
that the state focused on reorganizing how staff conduct their work by 
fostering collaboration among program officers. 

School districts — whose second greatest capacity challenge related to 
human capital — reported making great or very great effort to build human 
capital capacity for RTT reform by training existing staff (80 percent), 
expanding the responsibilities of current staff (74 percent), and shifting 
responsibilities among staff (64 percent). Similarly, all three district 
officials we spoke with in our follow-up interviews noted that efforts to 
build human capital capacity focused on training and shifting the roles of 
their current staff. One district official explained that they avoided funding 
new staff positions that they might not be able to retain after RTT funds 
ended. To build financial capacity, an estimated 23 percent of currently 
participating districts reported receiving supplemental funding from their 
state general fund. Additionally, an estimated 7 percent of districts 
reported receiving funds from foundations to build capacity. 

Despite their efforts, state and district officials reported that capacity 
struggles would likely remain once the RTT grant period ends. For both 
states and districts, financial capacity and human capital capacity 
represented the greatest challenges to sustaining reforms (see fig. 4). 
However, states and districts also reported planning to take various 
actions to help sustain their capacity for reform. All 19 states, as well as 
an estimated 84 percent of currently participating districts, indicated that 
retaining staff with requisite knowledge and skills is part of their plan to 
sustain RTT reform efforts. For example, one district official explained 
that they used a large portion of their RTT funds on training for teachers 
and administrators. Using the RTT funds for this purpose — as opposed to 
hiring many new staff — helped them build capacity and institutional 
knowledge that would be easier to sustain once the RTT funding ends. 
Additionally, 17 states indicated that modifying existing staff roles and 
responsibilities was the second most planned action to sustain RTT 
reforms. An estimated 72 percent of districts indicated that building 
institutional knowledge was their second most planned action to sustain 
RTT reforms. 
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Figure 4: Types of Capacity That Wiii Be Most Chaiienging to Sustaining Race to 
the Top Reforms, as Reported by States and Estimated for Districts 


Financial 

ntlT^ 

Human capital 
Stakeholder 


Organizational 



1 2 
Average challenge score 


5 



States 

Districts 


Source; GAO survey of RTT states and currently participating districts. | GAO-15-295 

Note: We asked states and districts to rate — on a scale of 1 to 5, where 1 is "not at all challenging" 
and 5 is "extremely challenging” — how challenging it would be to sustain the four types of capacity. 
Average challenge scores indicate the extent to which each capacity type would be challenging to 
sustain. Estimated average scores for the districts have 95 percent confidence intervals of within +/- 
0.1 3 of the estimated score. 


Rural Districts Faced 
Greater Capacity 
Challenges Than Urban 
and Suburban Districts in 
Many of the Reform Areas 


Rural school districts reported facing significantly greater challenges than 
urban districts in the standards and assessments and data systems core 
reform areas when implementing RTT, according to our survey results 
(see fig. 5).^^ 


Rural districts also reported facing statistically significantly greater challenges than 
suburban districts with stakeholder capacity in the standards and assessments reform 
area, and with organizational capacity in the effective teachers and leaders reform area. 
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Figure 5: Estimated Percentage of Rurai Districts Reporting Each Type of Capacity 
as Very or Extremeiy Chaiienging in impiementing Race to the Top Compared to 
Urban Districts, by Reform Area 
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Sources: GAO survey of RTT currently participating districts; images (GAO and Art Explosion). | GAO-1 5-295 

Note: Percentage estimates have 95 percent confidence intervals of within +/- 10 percentage points 
of the estimated percent, and differences between rural and urban percentages are statistically 
significant. 


These survey results are consistent with our past work on the capacity 
challenges rural districts face. For example, in a 2013 report, we found 
that a rural district in New York faced unique difficulties implementing its 
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teacher evaluation system because its small student population required 
some teachers to teach more than one subject, which made the 
evaluation process more complex and time-consuming.^® Similarly, our 
prior work on implementation of School Improvement Grants showed that 
rural districts faced difficulties because attracting and retaining high- 
quality teachers and implementing increased learning time requirements 
were difficult, in part due to higher transportation costs in rural areas. 

In addition, in responding to our survey, rural districts reported 
anticipating more difficulty than urban districts in sustaining all four types 
of capacity after the RTT grant period ends; and anticipated more 
difficulty than suburban districts in sustaining three of the four capacity 
types. For example, according to our survey, an estimated 40 percent of 
rural districts anticipated that human capital capacity would be very or 
extremely challenging in sustaining RTT reform efforts compared to 26 
percent for urban and suburban districts (see fig. 6). One expert 
participating on our panel agreed, noting that rural districts would also 
face challenges sustaining reforms because constrained budgets and a 
lack of human capital capacity are often particularly challenging for rural 
districts. In addition, a rural district official told us that they have a small 
number of employees, and attracting and retaining skilled employees who 
can perform multiple work functions can be more difficult for them. The 
official also noted that recruiting staff is a challenge because rural districts 
are often also among the poorer districts and do not have the resources 
to implement large-scale hiring efforts. 


GAO-1 3-777. 

GAO, School Improvement Grants: Early Implementation Under Way, but Reforms 
Affected by Short Time Frames, GAO-1 1-741 (Washington, D.C.: July 25, 201 1 ). See also 
GAO-12-373. 
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Figure 6: Types of Capacity That Wiii Be Most Chaiienging to Sustaining Race to 
the Top Reforms, as Reported by States and Estimated for Rurai Districts, 
Compared to Urban and Suburban Districts 



Sources: GAO survey of RTT currently participating districts; images (GAO and Art Explosion). | GAO-15-295 

Note: All percentages have 95 percent confidence intervals of within +/- 10 percentage points of the 
estimate itself. Within each type of capacity challenge, differences exceeding 10 percentage points 
are significant. 


Although states and districts across the country likely face capacity 
challenges and resource limitations to some degree, research suggests 
that some rural districts — and states that have many rural districts — may 
be less likely to have the skills, knowledge, or expertise to overcome 
these challenges. For example, one 2013 report recommended that 
states may have to play a much more direct role in guiding school 
improvement in smaller, rural districts, where capacity is lacking.^® In 
addition, a 2014 Education Office of Inspector General report indicated 


Ashley Jochim and Patrick Murphy, The Capacity Challenge: What It Takes for State 
Education Agencies to Support School Improvement (Seattle, WA: December 2013). 
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this approach may be effective in reducing project deiays and provided an 
exampie of a state that pianned to heip districts buiid capacity in order to 
better support iow-performing schoois in rurai areas. 

Our prior work and other research demonstrate that states with many 
rurai districts need additionai supports in this area. Given that rurai 
districts reported that they faced chaiienges impiementing and sustaining 
reforms that were statisticaiiy significantiy greater than urban and 
suburban districts, a greater understanding of these chaiienges couid 
heip Education provide more targeted support to rurai districts. According 
to Education’s Handbook for the Discretionary Grant Process, Education 
is to provide technicai assistance to grantees to heip them achieve 
successfui project outcomes. Education officiais toid us that they are 
aiso required to hoid grantees accountabie for meeting the commitments 
made in their approved RTT appiications. Education has recognized and 
reported on chaiienges facing rurai districts. in addition. Education 
officiais stated that they have supported RTT grantees and their rurai 
districts through a series of convenings, work groups, pubiications, 
webinars, and individuai technicai assistance, and provided exampies of 
these activities. However, we reviewed RSN’s technicai assistance 
documents and found that most of the activities were not provided in the 
manner that RTT states reported finding most heipfui — as discussed iater 
in this report — nor were they taiiored to heiping states address the unique 
capacity chaiienges that rurai districts reported facing in the reform areas 
identified in our survey. Uniess Education provides assistance specificaiiy 
designed to heip states support their rurai districts in addressing their 
capacity chaiienges in impiementing and sustaining high-quaiity reform, 
states may not be abie to heip the districts that need it the most. 


oq 

U.S. Department of Education Office of Inspector General, The Department’s 
Monitoring of Race to the Top Program Recipient Performance, ED-OIG/A19M0003 
(Washington, D.C.: January 3, 2014). 

U.S. Department of Education, Handbook for the Discretionary Grant Process, 
Handbook OS-01 (Washington, D.C.: January 2009). 

41 

U.S. Department of Education, Institute of Education Sciences, National Center for 
Education Evaluation and Regional Assistance, A Focused Look At Rurai Schoois 
Receiving Schooi improvement Grants: Evaluation 6r/ef (Washington, D.C.: April 2014). 
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Many RTT States 
Found Education’s 
Technical Assistance 
the Most Helpful 
Resource 


States Found Technical 
Assistance Most Helpful in 
Building and Sustaining 
Capacity 


According to our state survey, individualized technical assistance 
provided by Education program officers was the most helpful resource 
when building capacity to implement and sustain reform plans (see fig. 7). 
This was consistent with the views of officials we interviewed in four RTT 
states, who described very positive interactions with their Education 
program officer. For example, state officials explained that the program 
officers practiced collaborative problem-solving and provided a significant 
amount of support to the state as it implemented reform activities. 


The next most helpful resources, according to our state survey, were 
technical assistance provided by other staff in the Implementation and 
Support Unit and RSN. One state official we spoke with noted that 
Implementation and Support Unit staff provided useful information on how 
other states were implementing their reform activities. An official from 
another state explained that the state is working closely with RSN to 
better understand how to work with its participating RTT districts to better 
leverage federal funding to improve student outcomes. 
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Figure 7: Extent to Which Race to the Top (RTT) Resources Were Heipfui in 
Buiiding Capacity to impiement and Sustain Reform, as Reported by States 
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Implementation and Support Unit staff 

Individualized Technical Assistance 
from the Reform Support Network 

Reform Support Network 
Communities of Practice 

Reform Support Network Resources 
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Not at all/somewhat helpful 
Moderately helpful 
Very/extremely helpful 


Source: GAO survey of RTT states. | GAO-15-295 

Note: We asked states to rate — on a scale of 1 to 5, where 1 is “not at all helpful’’ and 5 is “extremely 
helpful” — how helpful assistance from Education and the Reform Support Network were to building 
capacity to implement and sustain RTT reforms. Scores indicate how helpful each RTT resource was 
to states. 


As shown in figure 7, RSN’s communities of practice ranked fourth in 
terms of helpfulness to build capacity to implement and sustain RTT 
reform. According to state officials and one expert participating on our 
panel, these communities of practice encouraged collaboration across 
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states, which has helped them leverage knowledge, talent, and 
resources, as well as facilitate the sharing of promising practices. 
Education officials observed similar value in RSN’s communities of 
practice, noting that through them, states had a forum in which to learn 
from each other and discuss RTT implementation issues. It is worth 
noting that state officials we interviewed commented that communities of 
practice may have been more helpful to states that were in the early 
stages of implementing RTT reforms. For example, one official noted that 
their state was farther along in implementing its teacher and principal 
evaluation system and school turnaround efforts and therefore did not 
gain as much from those communities of practice. 

State officials ranked RSN’s capacity-building community of practice and 
web-based resources from Education and RSN among the least helpful to 
states. Education officials similarly noted that while webinars were an 
easy way to disseminate information, they are likely not as valuable as 
other RTT resources because they are not as tailored to a particular 
state’s needs. Two experts participating on our panel noted that although 
an abundance of school reform-related information exists on websites, 
little is known about the effectiveness of the information. 

In December 2013, RSN published the results of an evaluation of its 
technical assistance activities that generally aligned with the results of our 
state survey. For example, according to RSN’s evaluation report, 
participants indicated they were satisfied with the quality of the support, 
the format and content of the technical assistance activities provided by 
RSN. Individualized technical assistance had the highest ratings because, 
according to the evaluation report, it was designed to address a state’s 
specific implementation challenges. In addition, participants in the RSN 
evaluation indicated that on average, technical assistance activities had a 
moderate effect on states’ ability to build capacity overall. The results of 
the RSN evaluation also showed that while webinars were useful for 
disseminating information to larger audiences and convening states on a 
regular basis, they received lower ratings than other forms of assistance. 
Our body of work on performance measures and evaluations has shown 
that successful organizations conduct periodic or ad hoc program 
evaluations to examine how well a program is working. These types of 


GAO, Performance Measurement and Evaluation: Definitions and Relationships, 
GAO-11-646SP (Washington, D.C.: May 201 1 ); GAO, Designing Evaluations: 2012 
Revision, GAO-12-208G (Washington, D.C.: January 2012). 
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evaluations allow agencies to more closely examine aspects of program 
operations, factors in the program environment that may impede or 
contribute to its success, and the extent to which the program is operating 
as intended. Information from periodic reviews of RSN’s technical 
assistance efforts are an important factor in determining if adjustments 
are needed to help grantees meet their goals for education reform. 

State officials we surveyed also identified additional activities that 
Education could undertake that would better assist states with 
implementing RTT. Specifically, 10 of 19 states reported wanting ongoing 
professional development throughout the grant period, as opposed to 
during the early stages of the grant. Ten of 19 states reported wanting 
training to be provided in their respective states to make it more easily 
accessible, rather than having to travel to Washington, D.C. Further, 11 of 
19 states reported wanting assistance identifying skilled contractors who 
could assist with reform efforts. Education officials stated that any 
assistance it provides to identify contractors cannot compromise the 
fairness and objectivity of the states’ procurement processes."^® Education 
officials also pointed out other legal challenges to identifying contractors, 
such as prohibitions against endorsements of private entities."^® However, 
Education officials stated they can assist grantees by, for example, 
helping them to develop objective criteria, analysis, or research regarding 
the qualifications of skilled contractors. They said they can also provide 
resource lists using objective criteria, as well as technical assistance in 
this area. 


Elements often examined in such evaluations include the program activities’ 
conformance with statutory and regulatory requirements, program design, and customer 
expectations. See GAO-1 1-646SP. 

Education officials consider professional development to be activities such as 
identifying strategies to improve instructional practices. 

Further, Education officials stated that if they held activities such as vendor fairs to help 
identify contractors, some states might not be able to attend or, by attending, they might 
be in danger of violating some of their state procurement standards. 

See, e.g., 5 C.F.R. § 2635.702. 
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Education Established a 
New Office and Aims to 
Develop Policies that 
Reflect a Coordinated 
Approach to Providing 
Technical Assistance 


In October 2014, Education created the Office of State Support to expand 
and sustain the collaborative approach to providing oversight and 
technical assistance that began under the Implementation and Support 
Unit. More specifically, the purpose of the Office of State Support is to 
design a coordinated approach across multiple Education programs to 
reduce redundancy and improve the efficiency and effectiveness of 
Education’s oversight efforts. The Office of State Support will provide 
states with one point of contact for multiple education programs that will 
provide support and technical assistance. The Office of State Support 
plans to establish advisory committees, involve staff from other education 
programs in decision making, and maintain close communication with 
staff from other education programs that have similar goals and activities 
as programs covered under the new office. 


Officials from the Office of State Support stated that the lessons learned 
from the RTT monitoring and technical assistance processes will inform 
their work in the new office for programs they oversee — many of which 
are helping states to facilitate comprehensive education reforms similar to 
those started under RTT. However, officials stated that they will need to 
eventually transition to a longer-range plan for monitoring and reconsider 
how they provide technical assistance because Education’s contract with 
RSN ends on June 30, 2015. Education officials noted that it was unlikely 
that the department would receive such a large amount of funding ($43 
million) for technical assistance again. They explained that the type and 
extent of technical assistance efforts to states after the end of the RSN 
contract will, in turn, be dependent upon the funding available for that 
purpose. Lastly, they said that they will look to leverage existing technical 


In addition to RTT, the Office of State Support administers Title I (including ESEA 
Flexibility), Title II, Title III, School Improvement Grants, RTT Assessment, and the 
Comprehensive Centers program. Title I, Part A of ESEA, as amended, provides financial 
assistance to districts and schools with high numbers or percentages of children from low- 
income families in part to help ensure that all children meet challenging state academic 
standards. Title II, Part A of ESEA provides grants to state educational agencies, districts, 
and other entities to increase student academic achievement through strategies such as 
improving teacher and principal quality and hold districts and schools accountable for 
improvements in student academic achievement. Title III of ESEA provides funding to 
support programs for English language learners. The RTT Assessment Program provided 
funding to consortia of states to develop computer-based statewide assessments aligned 
to common college- and career-ready standards. The Comprehensive Centers program 
awards discretionary grants that support 22 comprehensive centers to help increase state 
capacity to assist districts and schools in meeting student achievement goals. 
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assistance funds, such as those provided for the Comprehensive Centers 
program, to help increase state capacity to assist districts and schools. 

Education’s Handbook for the Discretionary Grant Process requires 
program offices to develop a monitoring and technical assistance plan for 
each grant program."^® In addition, according to Federal Standards for 
Internal Control, policies and procedures help ensure that necessary 
actions are taken to address risks to achieving the entity’s objectives.'^® 
Education has a monitoring and technical assistance plan for RTT, which 
it has been using for the past four years and has continued to use during 
the transition from the Implementation and Support Unit to the Office of 
State Support. However, officials from the Office of State Support stated 
that they planned to establish coordinated technical assistance processes 
and procedures for all of the programs administered by the new office, 
while meeting the needs of the states and their particular initiatives. For 
example, they said they need to consider how to bring the various kinds 
of monitoring and technical assistance conducted by different program 
offices together to provide support for and make connections across 
programs, and be less burdensome for states. Officials stated that they 
formed a working group of staff from various Education program offices, 
including former Implementation and Support Unit staff, to help inform the 
new office’s coordinated technical assistance policies. However, officials 
noted that the working group was in the early stages of this process, and 
had not yet developed any draft policies or established a definitive 
deadline for accomplishing this task. 

Given the valuable technical assistance that RSN provided to states, and 
that Education has not determined the type or amount of technical 
assistance to be provided, there could be a gap in the type of support that 
Education can provide to states when the contract expires. Until the 
Office of State Support develops and finalizes policies and procedures 
that include support activities states identified as most helpful. Education 


4ft 

U.S. Department of Education, Handbook for the Discretionary Grant Process, 
Handbook OS-01 (Washington, D.C.: January 2009). The monitoring and technical 
assistance plan serves as a standard and guide for monitoring grants in the program. 
Such a plan includes: (1) purpose, goals, and objectives; (2) performance measures and 
data; (3) grantee performance; (4) grantee or project risk factors; and (5) monitoring and 
technical assistance. 

GAO, Standards forinternai Controi in the Federai Government, GAO/AIMD-00-21.3.1 
(Washington, D.C.: November 1999). 
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runs the risk of not providing the most effective assistance to its grantees 
to help them successfully implement and sustain reform efforts. 


Experts Identified 
Lessons Learned to 
Help Sustain Reform 
and Made 
Observations to 
Consider in Future 
Efforts 


Our analysis of our expert panel transcript revealed key lessons that 
could help states and districts address their greatest capacity challenges 
and help sustain reforms after the RTT grant period ends. 

To address challenges with financial capacity, five of the 10 experts 
participating on our panel noted that federal formula grants are better 
suited than competitive grants for building and sustaining capacity 
because they provide a more stable funding source. Three experts 
stated that there are several ways that states and districts can leverage 
the funds they receive annually in formula grants to help sustain 
reforms.^'' The Title I formula grant — designed to improve schools with 
high concentrations of students from low-income families — gives districts 
and schools flexibility to use federal funds to support instructional 
strategies and methods that best meet local needs. For example, schools 
where at least 40 percent of students are from low-income families may 
operate “school-wide” Title I programs, which allow schools to combine 
Title I funds with other federal, state, and local funds to improve the 
overall instructional program for all children in a school. In the 2012-2013 


Key Lessons Learned 
from RTT May Be Critical 
to Sustaining Reform 


We performed a content analysis of the transcript of the panel discussion to develop 
common themes among the experts on lessons learned from RTT that could help sustain 
reform efforts and inform future education reform efforts. We tallied responses for each 
panelist who commented on those themes. 

These include grants under Title I, Part A (Title I); Title II, Part A (Title II) of ESEA, as 
amended; and Part B of the Individuals with Disabilities Education Act (IDEA). 
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school year, approximately 40,632 schools, or 74 percent of all Title I 
schools, operated school-wide programs. 

Despite the large number of schools running a school-wide program, 
districts and schools may not be using the flexibilities to combine Title I 
funds with other federal funds to their fullest extent due, in part, to a lack 
of organizational capacity at the state and district levels. According to 
Education officials and two experts on our panel, states and districts are 
often uncertain about whether they are allowed to combine federal 
formula grants in new ways to support comprehensive reforms. For 
example. Education officials told us that historically, states and districts 
have used Title II funds — formula grants designed in part to increase 
student academic achievement through strategies such as improving 
teacher and principal quality — to reduce class size. However, according 
to Education’s guidance, states and districts could also choose to 
combine Title I and Title II funds to sustain reforms initiated under RTT, 
such as providing academic support coaches and financial incentives and 
rewards to attract and retain qualified and effective teachers to help low- 
performing schools. According to five experts on our panel, 
uncertainties about what is allowed may stem from lack of communication 
and coordination among the multiple federal education program and 
financial management offices, and because these offices are not always 
focused on helping states and districts better leverage their funds. 

According to Federal Standards for Internal Control, information should 
be communicated in a form and within a time frame that enables an 
agency to achieve its objectives. “ In 201 1 and 2013, Education provided 
guidance to states clarifying ways to leverage federal formula grant 


For more examples of comprehensive school-wide reform strategies that could be 
supported by Title I and Title II, see U.S. Department of Education guidance, Using Title I, 
Part A ARRA Funds for Grants to Local Educational Agencies to Strengthen Education, 
Drive Reform, and Improve Results for Students, September 2, 2009. 

“ GAO/AIMD-00-21.3.1. 
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programs to support the four core reform areas. Further, in 2013, the 
Council of Chief State School Officers developed a toolkit for states to 
help clarify how districts and schools may spend K-12 federal formula 
grants. This toolkit encourages states to improve collaboration among 
offices supported by federal grants to help ensure they effectively 
leverage federal funds. Currently, Education is working with RSN to 
develop another toolkit for states and districts on ways to leverage federal 
formula grants to sustain educational reforms. Education officials could 
not provide definitive time frames for the release and dissemination of the 
toolkit, but noted that they are hoping to release it sometime in 2015. This 
toolkit, when finalized, may help states and districts better understand 
how to leverage their formula grants to sustain reform activities and help 
raise student achievement — a primary objective of education reform. 

Education officials and one expert participating on our panel also said that 
states and districts do not use funding flexibilities to their fullest extent 
because they have concerns about compliance with state audit 
requirements. Education officials explained that states and auditors may 
believe that federal law prohibits certain activities, even when the law and 
its implementation rules do not. Education officials told us they tried to 
address these uncertainties by issuing guidance to clarify how states and 
districts can leverage federal funds to support reforms. According to this 
guidance, states may use Title I funds to provide technical assistance to 
low-achieving schools, and districts may consolidate Title I, Title II, and 
IDEA funds in schools under the school-wide program to support 
comprehensive reforms by, for example, extending the school day or 
school year.®® However, Education officials said that there is still 


See U.S. Department of Education guidance, Using Title I, Part A ARRA Funds for 
Grants to Local Educational Agencies to Strengthen Education, Drive Reform, and 
Improve Results for Students. See also U.S. Department of Education letter to Governors, 
Flexibility in Using Federal Funds to Meet Local Needs, March 3, 201 1 . 
http://www2.ed.gOv/policy/gen/guid/secletter/1 1 0303.html and Maximizing Flexibility in the 
Administration of Federal Grants, IDEA, Title I, Title II, and Non-Federal Funds in 
Schoolwide Programs, March 25, 2013. 
www2.ed.gov/programs/titleiparta/flexswp091 31 3.pdf 

®® Council of Chief State School Officers, Maximizing Federal Education Funds for 
Student Achievement: A Toolkit for States Seeking to Enhance Flexibility and Reduce 
Burden, November 201 3. 

®® Maximizing Flexibility in the Administration of Federal Grants, IDEA, Title I, Title II, and 
Non-Federal Funds in Schoolwide Programs, March 25, 2013. 
www2.ed.gov/programs/titleiparta/flexswp091 31 3.pdf 
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confusion about this issue, particularly among the audit community, and 
that it needs to provide new guidance to help auditors better understand 
allowable spending within federal formula grants, especially with Title I 
funds. However, it does not have a definitive plan for developing and 
implementing this guidance. Such guidance — when developed and fully 
implemented — may help auditors better understand funding flexibilities in 
existing formula grants and help states and districts fully leverage these 
flexibilities. 

Further, the pending reauthorization of ESEA also provides an 
opportunity to address these capacity issues. Education told us that it is 
exploring new options to help states and districts build capacity to 
implement comprehensive reforms, including increasing the portion of 
Title I grant funds that can be set aside for administrative purposes.®^ 
Currently, two of the set-asides in the Title I program limit the maximum 
percentage of funds that can be set aside to support state administrative 
functions and districts’ school improvement activities. Specifically, ESEA 
requires that a state generally spend no more than 1 percent (or 
$400,000, whichever is greater) of its Title I funds on state 
administration®® and 4 percent on district school improvement activities.®® 
Education told us that the current portion of funds under the ESEA Title I 
grant that may be used for administrative functions may be inadequate 
given the range and complexity of state-level work in supporting effective 
implementation of local Title I projects. In its fiscal year 2016 budget 
proposal, the Administration proposed increasing the funds a state can 
spend on administration from 1 percent to 3 percent. According to 
Education officials, the trade-off, particularly in a tight fiscal environment, 
is that larger set-asides may reduce the portion of available funds that 
would transfer to districts and schools to implement programs. In the 
current Congress, the Student Success Act, which was reported out of 


®^ States and districts can designate or “set aside” a portion of Title I grants for certain 
required and optional purposes before the remaining funds are transferred to districts and 
schools. 

®® To carry out administrative duties under parts A, C, and D of Title I, a state may 
generally reserve 1 percent of the amounts it receives under those parts or $400,000, 
whichever is greater. 20 U.S.C. § 6304. 

®® To carry out various duties related to school improvement, a state is required to reserve 
4 percent of the amount it receives under subpart 2 of Title I part A. 20 U.S.C. § 6303. The 
amount states reserve under this section cannot reduce grants to school districts under 
Title I, part A, subpart 2 below the amount received in the prior year. 20 U.S.C. § 6303(e). 
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the House Committee on Education and the Workforce, would make 
changes to both of these set-asides. 

To help address human capital and stakeholder capacity challenges, five 
experts on our panel noted the importance of fostering partnerships 
between a state and its districts, among districts within a state, and with 
non-governmental entities by, for example, convening groups of experts 
across the state to share expertise, solve problems, and share lessons 
learned to help leverage knowledge and talent. They further noted the 
potential for such a strategy to solve common challenges, such as how to 
develop effective strategies for evaluating teachers who teach subjects 
that are not assessed using standardized tests (e.g. foreign language or 
art). Universities with research and professional development institutes 
are another potential resource to help states and districts build and 
sustain human capital capacity. For example, one expert noted that 
strong relationships with higher education institutions and teacher unions 
are needed to revamp teacher, principal and superintendent training 
programs and teacher licensure requirements. Lastly, three panelists said 
that to maintain key stakeholder support for reforms, states need to show 
progress in meeting their established time frames for RTT reform, or 
increase student achievement. 


Observations from Experts 
May Help Education 
Address Capacity 
Concerns in Future Grant 
Competitions 


Three experts on our panel noted that competitive grants may be better 
suited than formula grants for spurring reforms and innovative 
approaches, but varying levels of capacity among states and districts 
raises concerns about their ability to win competitive grants and 
successfully implement large-scale education reforms. Research 
suggests that states’ capacity was an important variable in helping to 
predict who applied for RTT funds and which states scored well during 
the competition. In particular, a 201 1 study found that states with quality 
standards and accountability procedures, and that had achieved overall 
student gains, were more likely to receive higher scores during the RTT 
grant competition.®^ When making competitive grant awards in the future. 


H.R. 5, 114th Cong. (2015). For school improvement, this bill would change among 
other things the percentage of the set-aside to 7 percent, and that percentage would be 
taken from funds received under the newly amended part A, subpart 1 , chapter B. For 
state administration, the percentage and dollar figure would remain the same, but the 
amount would be taken from funds received under newly amended Title I, part A, subparts 
1,2, and 3. 

Manna and Ryan, “Competitive Grants and Educational Federalism,” p. 541. 
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Education officials told us they expect to look at demonstrated capacity as 
evidenced by a state’s performance under previous grants and may offer 
a competitive priority for previous success. To help states and districts 
that may be struggling in these areas, experts participating on our panel 
made four observations that they believe could be incorporated into the 
design of future competitive grants to help level the playing field between 
high- and low-capacity states and districts. Education has incorporated 
some of the observations into its competitive grant programs to varying 
degrees and pointed out some advantages and disadvantages of each. 

• Observation 1 : Allow joint applications so that states and 
districts with greater capacity can partner with those with less 
capacity. Education noted that it used this approach in recent grant 
competitions. Education encouraged states that opted to adopt a 
common set of college- and career-ready standards to form 
collaborative groups to apply for RTT assessment grants to develop 
assessments aligned with the new standards.®^ A 201 1 study 
proposed that such arrangements could help states with less capacity 
more easily benefit from the initiatives of ones with more capacity by 
helping them identify partners and providing them access to funds that 
may help valuable reforms gain traction.®® Education officials told us, 
however, that when they have allowed joint applications or consortia 
for some competitive grants, the complexity of implementing the 
grants increased because states have different procurement rules 
which take longer to navigate. Education officials also noted that 
these joint initiatives sometimes take longer to implement because 
states have to establish a framework for how they are going to 
coordinate. 

• Observation 2: Staggering or “phasing” competitive grant 
funding to aiiow for varying capacity needs of grantees. 

Education officials told us that they have had mixed success using 
planning grants to allow grantees additional time to build capacity to 
implement plans. For example. Education used a two-phase strategy 


®® Authorized under the Recovery Act, the RTT Assessment Program provides funding to 
consortia of states to develop assessments that are valid, support and inform instruction, 
provide accurate information about what students know and can do, and measure student 
achievement against standards designed to ensure that all students gain the knowledge 
and skills needed to succeed in college and the workplace. 

®® Manna and Ryan, “Competitive Grants and Educational Federalism,” p. 543. 
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for awarding competitive grants under its Promise Neighborhoods 
grant program, including 1-year planning grants to organizations to 
enhance the grantees’ capacity and a separate competition for a 5- 
year implementation grant to organizations that demonstrated they 
were ready to implement their plans. However, we recently reported 
that Education did not communicate clearly to grantees about its 
expectations for the planning grants and the likelihood of receiving 
implementation grants.®® Education officials told us that they do not 
always have the authority to offer this feature, but they consider it 
where it is possible. Education officials told us that they are 
considering adding a planning year to the School Improvement Grant, 
which is federal money awarded to states that states, in turn, award to 
districts using a competitive process. Education officials told us that 
they believe that low-capacity districts could benefit from this 
approach, but noted that it will be important to emphasize their 
expectation that grantees use the planning year to build capacity to 
implement their reform plans. 

• Observation 3: Allowing intermediary entities that often help 
coordinate or provide technical assistance to districts to apply 
for competitive grants. Education officials told us that they see a 
benefit to using partners such as nonprofit organizations to drive 
reform, noting, for example, that the Investing in Innovation program 
allows nonprofits to partner with school districts as part of the 
application process and throughout the grant period.®® Research 


®^ Since 2010, Education has competitively awarded Promise Neighborhoods planning 
and implementation grants to 48 community-based organizations, including nonprofits, 
institutions of higher education, and Indian tribes that work in partnership with several 
other organizations, such as schools and social service agencies. Promise Neighborhoods 
grants are one of several place-based strategies under the White House Neighborhood 
Revitalization Initiative, which was also launched in 2010. 

®® GAO, Education Grants: Promise Neighborhoods Promotes Collaboration but Needs 
National Evaluation Plan, GAO-14-432 (Washington, D.C.: May 5, 2014). 

®® The Investing in Innovation Fund, established under section 14007 of the Recovery Act, 
provides funding to support (1 ) local educational agencies and (2) nonprofit organizations 
in partnership with (a) one or more local educational agencies or (b) a consortium of 
schools. The purpose of this program is to provide competitive grants to applicants with a 
record of improving student achievement and attainment in order to expand the 
implementation of, and investment in, innovative practices that are demonstrated to have 
an impact on improving student achievement or student growth, closing achievement 
gaps, decreasing dropout rates, increasing high school graduation rates, or increasing 
college enrollment and completion rates. 
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supports such an approach as well. A 201 1 RAND study examining 
the federal and state role in improving schools in 15 states found that 
although some states assumed primary responsibility for assisting 
low-performing schools, others relied on regional organizations, area 
education agencies, or intermediate school districts to fill this role.®^ 
However, Education officials noted that applicant eligibility is generally 
defined in statute. 

• Observation 4: Streamlining Education’s grant application 
processes to make it easier for states and districts with less 
capacity to apply. Education officials told us that one example of 
streamlining the grant process was allowing states that did not win an 
award in the first phase of a competition to revise the same 
application and resubmit for subsequent phases. Education adopted 
this strategy in the RTT grant competition. Another way to streamline 
the grant application process is by encouraging shorter applications. 
Education officials said it used this approach in a grant competition for 
the Investing in Innovation program. Education officials noted that, in 
general, one disadvantage to shorter applications is that there may 
not be sufficient detail in the applications to hold grantees accountable 
for implementing their plans. 


Conclusions 


As Education’s technical assistance contract for RSN comes to a close, 
and it develops new processes for technical assistance under the new 
Office of State Support, it has an opportunity to apply the technical 
assistance that RTT states reported as most helpful, such as 
individualized technical assistance and professional development, to 
other grant programs that the office oversees. Such technical assistance 
could help states implement and sustain the comprehensive education 
reforms which will continue to be supported by other grant programs 
managed by the Office of State Support. 

In addition, because rural districts face unique challenges implementing 
and sustaining RTT reforms, focusing efforts to enhance Education’s 
understanding of the types of additional supports they may need could 
help these districts successfully implement and sustain their reform 
efforts, and ultimately improve student achievement. Further, as the RTT 


Gottfried, Michael A., Brian M. Stecher, Matthew Hoover, and Amanda Brown Cross, 
Federal and State Roles and Capacity for Improving Schools, RAND Corporation 
(Pittsburgh, PA: 2011). 
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grant period comes to an end, RTT states may need to better leverage 
their federal formula grants to continue to support comprehensive reform 
in the absence of RTT funds. Education officials and other experts have 
emphasized the importance of leveraging existing funding flexibilities in 
education formula grants to help states implement and sustain large-scale 
reform efforts. However, concerns about a lack of communication 
between states’ program and financial management offices, as well as 
concerns about non-compliance with state and federal requirements may 
be limiting states’ willingness to use the funding flexibilities present in 
current law to develop and implement strategies tailored to their unique 
local needs. By taking actions to address these issues. Education can 
help states and districts better use their federal funding in the most 
effective way to improve student achievement and to support 
comprehensive school reform. 


Recommendations 


To help ensure that states are better able to sustain RTT reforms and that 
Education can effectively support other grant programs managed by the 
Office of State Support, we recommend that the Secretary of Education 
direct the Office of State Support to fully implement and incorporate into 
its coordinated technical assistance policies and procedures the types of 
support that would be useful in sustaining RTT reforms and providing 
effective support to grantees in other programs supporting education 
reform that the Office of State Support oversees. These could include: 

• providing individualized technical assistance to states, such as that 
currently provided by Education program officers; 

• facilitating communities of practice to promote opportunities for 
collaboration across states; 

• providing professional development (or training) throughout the grant 
period, as opposed to only during the early stages of the grant; 

• making training more easily accessible by conducting training locally 
in their respective states, when possible; and 

• to the extent permissible in the context of federal and state 
requirements and restrictions, exploring the possibility of assisting 
states in identifying skilled contractors to help implement reform 
efforts. 

To help states address capacity challenges as they sustain 
comprehensive education reforms similar to RTT, we recommend that the 
Secretary of Education direct the Office of State Support to take steps, 
such as: 
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• providing ongoing individualized technical assistance to states to help 
them target assistance to rural districts, particularly in the reform 
areas that were most challenging for rural districts; 

• finalizing and disseminating guidance to be included in Education’s 
toolkit to help states leverage federal formula grants to sustain 
education reforms; and 

• clarifying and improving understanding of how funding flexibilities in 
existing formula grants could be used to support education reform 
efforts to help states and the audit community address impediments to 
using formula grants in different ways. 


Agency Comments 
and Our Evaluation 


We provided a draft of this report to the Department of Education for 
comment. Education provided technical comments, which we 
incorporated into the report as appropriate. Education’s written comments 
are reproduced in appendix VI and summarized below. Education did not 
explicitly agree or disagree with our recommendations, but outlined steps 
to address many elements contained in them. It also provided additional 
information related to our findings and recommendations. 

In response to our first recommendation. Education stated that it shares 
our interest in supporting states as they sustain RTT reforms and 
supporting other grant programs under the Office of State Support 
through performance management and technical assistance. To this end. 
Education described plans to build on its generally successful RTT 
monitoring strategy to develop a consolidated technical assistance 
strategy for all programs under the auspices of the Office of State 
Support. We have added clarifying language in the body of the report to 
better reflect existing elements of the RTT monitoring and technical 
assistance plan. 

Education’s plan to provide coordinated policy development, performance 
management, technical assistance, and data analysis services through a 
structure intended to more effectively support the implementation of key 
reforms and provide individualized support is a positive step. These 
coordinated policies and procedures could continue to support RTT 
grantees as well as other grantees under other Office of State Support 
programs that have a role in helping states implement comprehensive 
education reforms. However, we continue to believe that until these 
policies are fully implemented. Education risks providing less effective 
support than it otherwise might. Further, as Education’s technical 
assistance contract for RSN comes to an end, we continue to believe that 
Education should take explicit steps to incorporate into its new 
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consolidated assistance strategy for all programs under the Office of 
State Support the technical assistance activities that RTT grantees 
identified as being most helpful to them in sustaining their reforms. In 
addition, Education should incorporate those additional supports that 
states reported as desirable. We have clarified the intent of our 
recommendation accordingly. 

In response to our second recommendation, Education agreed that it is 
important to identify ways to help states target assistance to rural districts. 
Education stated, however, that the draft report does not adequately 
recognize the actions it has taken to support RTT grantees in rural states 
and districts, and provided a list of 17 activities it has undertaken through 
RSN to support rural areas. We acknowledge Education’s efforts to 
provide support to rural areas and have incorporated additional 
information in the draft report, as appropriate, to reflect this. However, in 
further reviewing these 17 activities, we found significant limitations and 
believe our overall finding and corresponding recommendation is still 
warranted. Specifically: 

• Nearly all of the activities (16 of 17) were in the form of working 
groups, convenings, webinars, toolkits, and publications developed by 
the RSN, many of which were located on the RSN website. According 
to our survey of all 19 RTT states, web-based resources were among 
the least helpful to RTT states in building and sustaining the 
necessary capacity to implement reforms. Only one of the 17 activities 
provided individualized technical assistance which, according to our 
survey, was the most helpful form of assistance to RTT states. We 
realize that Education formed RSN to provide support in a variety of 
formats and agree that RSN has generally well supported RTT 
grantees. However, given the unique capacity challenges that rural 
districts face, we believe there is value in offering technical assistance 
tailored to the individual needs of rural areas. 

• According to our generalizable survey of districts that received RTT 
funds, rural districts faced statistically significantly greater challenges 
than urban districts in implementing reforms in two areas: standards 
and assessments and data systems. However, 14 of the 17 RSN 
activities focused on the other two reform areas (school turnaround 
and effective teachers and leaders). RSN’s efforts to focus resources 
on assisting states in implementing RTT reforms are important ones, 
and we believe that many states and districts may have benefitted 
from these efforts. However, in order to best support states that are 
working to implement and sustain reforms in their rural districts. 
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Education should target future support in the reform areas in which 
rural districts most struggled: standards and assessments and data 
systems. Accordingly, we modified our recommendation to clarify that 
Education should take steps to provide targeted assistance to states 
in those reform areas that we have identified as statistically 
significantly more challenging for rural districts. 

• Many of the activities undertaken to support rural districts were 
conducted in 2012 and 2013 (6 of the 11 that included specific dates) 
when states and districts were fully engaged in implementing RTT 
reforms. However, our survey of districts that received RTT funds was 
deployed from June through September 2014, and the results 
indicated that rural districts continued to face challenges long after 
they would have availed themselves of these resources. 

• Some of the activities (6 of 17) provided support that was not 
specifically tailored for rural districts; rather, it could be applied in 
rural, suburban, and urban school settings alike. We continue to 
believe that opportunities exist to help states better target support to 
rural districts. Without a better understanding of the unique capacity 
challenges that rural districts face, and a more focused approach to 
providing support. Education may not be able to help the states and 
districts that need it the most. 

Finally, Education recognized the importance of clarifying its guidance on 
the use of funding flexibilities and provided several examples of “Dear 
Colleague” letters it has provided to states. We referenced one of these 
letters in the draft of the report. We did not include the other two “Dear 
Colleague” letters (guidance related to leveraging federal funds to support 
school counselors and digital education) because they do not address the 
use of funding flexibilities in support of education reform initiatives, which 
was at the heart of our finding and corresponding recommendation. To 
address this apparent confusion we have clarified our recommendation 
accordingly. We noted in our report, and Education emphasized, that it is 
working with RSN to release new guidance in 2015 on ways to leverage 
federal grants to sustain educational reforms. However, as stated in our 
report. Education officials could not provide definitive time frames for the 
release and dissemination of the toolkit. We continue to believe that until 
this guidance is fully implemented, states and districts will continue to lack 
clarity on howto leverage their formula grants to sustain reform activities. 
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We are sending copies of this report to the appropriate congressional 
committees, the Secretary of Education, and other interested parties. In 
addition, the report will be available at no charge on GAO’s web site at 
http://www.gao.gov. 

If you or your staff should have any questions about this report, please 
contact me at (617) 788-0580 or nowickij@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made key contributions to 
this report are listed in appendix VII. 

Sincerely yours. 



Jacqueline M. Nowicki 
Director, Education, Workforce, 
and Income Security Issues 
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Appendix I: Objectives, Scope, and 
Methodology 


We framed our study of capacity challenges faced by states and districts 
implementing Race to the Top (RTT) reforms around three objectives: (1) 
What effect did RTT have on education reform, and what capacity 
challenges did states and districts face in implementing and sustaining 
RTT initiatives?; (2) How helpful was the assistance the U.S. Department 
of Education provided to states to build capacity to implement and sustain 
RTT reforms?; and (3) What lessons have been learned from RTT that 
could inform future education reform efforts? 

In addressing these objectives, we incorporated elements of “grounded 
foresight,” a methodological approach developed by GAO to examine 
future implications by identifying key trends, emerging challenges, and 
opportunities to inform government’s future role and responsibilities. 
According to GAO’s internal grounded foresight methodology paper, the 
heart of the proposed approach consists of three elements of grounding, 
designed to support GAO’s core values of integrity and reliability: (1) a 
strong factual-conceptual base, (2) one or more methods for discussing 
or anticipating the future, and (3) transparent communication of the 
outcomes. We developed a strong factual-conceptual base to assure that 
relevant trends and occurrences related to capacity issues and 
competitive grants are documented, recognized, and understood as part 
of the study. We reviewed and analyzed existing literature on capacity 
issues and competitive grants in K-12 education using GAO’s prospective 
evaluation synthesis approach.'' We examined the features of RTT, and 
reviewed findings from published reports to identify capacity challenges. 
We also deployed two web-based surveys of state educational agency 
and district officials; reviewed relevant federal laws, regulations, and 
guidance; and conducted interviews with a variety of federal, state, and 
local officials. We then convened a panel of experts who were 
knowledgeable about capacity issues and federal grants to obtain their 
views on the implications of capacity challenges on the sustainability of 
RTT reform efforts and potential future competitive grants. We made the 
results of the two web-based surveys publicly available to help ensure 
transparent communication of the capacity challenges states and districts 
reported facing. 


^ GAO, Prospective Evaluation Methods: The Prospective Evaluation Synthesis, 
GAO/PEMD-10.1.10 (Washington, D.C.: November 1990). 
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Surveys of RTT Grantees 
and Districts 


Appendix I: Objectives, Scope, and 
Methodology 


To obtain information on capacity challenges states faced in 
implementing and sustaining RTT reforms we conducted a web-based 
survey of RTT points of contact at each state educational agency in all 19 
grantee states.^ We conducted the survey from May through July 2014. In 
the survey, we asked RTT states about their capacity to implement RTT 
efforts, the support received to do so, and efforts to build and sustain 
capacity for RTT reform, among other things. We received responses 
from all 19 RTT states for a 100 percent response rate. We reviewed 
state responses and followed up by telephone and e-mail with selected 
states for additional clarification and context. We also published survey 
responses in an e-publication supplemental to this report, RACE TO THE 
TOP: Survey of State Educational Agencies’ Capacity to Implement 
Reform (GAO-1 5-31 6SP, April 2015). 

To obtain information on capacity challenges districts faced in 
implementing and sustaining RTT reform efforts we conducted a web- 
based survey of a sample of district officials whose districts received RTT 
funds. ^ We selected a stratified random sample of 643 from 3,251 school 
districts that received RTT funds from a population of 18,541 school 
districts in the 19 RTT states (see table 1)."^ Although the focus was on 
districts that currently receive RTT funds, we also included districts that 
initially were participating in RTT but later decided to formally withdraw.® 


^ The 19 RTT grantee states were Arizona, Colorado, Delaware, District of Columbia, 
Florida, Georgia, Hawaii, Illinois, Kentucky, Louisiana, Maryland, Massachusetts, New 
Jersey, New York, North Carolina, Ohio, Pennsylvania, Rhode Island, and Tennessee. For 
the purposes of this report, we consider the District of Columbia a state. 

® School district officials who received the survey were either district superintendents or 
another point of contact in the district for the RTT grant program suggested by state 
educational agency contacts or based on available contact information. 

We stratified our sample based on size, urban status, and participation status in the RTT 
grant. Because of the unique management and reporting structure for the New York City 
Geographic Districts, we collapsed these into a single district. We also excluded four 
districts which had either closed, were administrative without any students, or had merged 
with another district and reported that they did not have any students. Finally, we excluded 
the single school district that comprises all of Hawaii because we conducted a separate 
state survey that would include those results. See GAO, Race to the Top: Survey of State 
Educational Agencies’ Capacity to Implement Reform (GAO-1 5-31 6SP, April 2015), an e- 
supplementto GAO-15-295. 

® We included school districts participating and receiving RTT funds at the time of our 
audit work and those who had withdrawn from RTT at the time of our audit work but had 
received RTT funds at some point. 
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We obtained data from Education’s National Center for Education 
Statistics, which maintains the Common Core of Data for public school 
districts, for the 201 1-12 school year. Our sample allowed us to make 
estimates to all RTT districts and to subpopulations by urban status of the 
district. 


Table 1: Description of the Population and Sample of Race to the Top (RTT) 
Districts 

Stratum 

Population/Universe 

Sample 

25 Largest Districts® 

24 

24 

Not a Current RTT District® 

254 

136 

City 

511 

127 

Suburban 

885 

160 

Rural 

1577 

196 

Total 

3251 

643 


Source: GAO survey of RTT districts | GAO-1 5-295. 

Notes: 

®We excluded the single school district that comprises all of Hawaii because we conducted a separate 
state survey that would include those results. 

‘’Districts were placed in this stratum if they were initially participating in RTT but later decided to 
formally withdraw. However, when some of these districts responded to our survey, they indicated 
they were currently participating in RTT. 


We conducted the school district survey from June through September 
2014 and had a 76.7 percent final weighted response rate. Because we 
followed a probability procedure based on random selections, our sample 
is only one of a large number of samples that we might have drawn. 

Since each sample could have provided different estimates, we 
expressed our confidence in the precision of our particular sample’s 
results as a 95 percent confidence interval (e.g., plus or minus 6 
percentage points). This is the interval that would contain the actual 
population value for 95 percent of the samples we could have drawn. 
Unless otherwise noted, all percentage estimates in this report have 
confidence intervals within plus or minus 6 percentage points. For other 
estimates, the confidence intervals are presented along with the 
estimates themselves. In the survey, we asked questions about school 
districts’ capacity to implement RTT efforts, the support received to do so, 
and efforts to build and sustain capacity for RTT reform, among other 
things. We reviewed survey responses and followed up by telephone and 
e-mail with selected districts, as needed for additional clarification and to 
determine that their responses were complete, reasonable, and 
sufficiently reliable for the purposes of this report. We also published 
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survey responses in an e-publication supplement to this report, RACE TO 
THE TOP: Survey of School Districts’ Capacity to Implement Reform 
(GAO-15-317SP, April 2015). 

The quality of the state and district survey data can be affected by 
nonsampling error, which includes variations in how respondents interpret 
questions, respondents’ willingness to offer accurate responses, and data 
collection and processing errors. To minimize such error, we included the 
following steps in developing the survey and in collecting and analyzing 
survey data. We pretested draft versions of the instrument with state 
educational agency officials in three states and officials in four districts to 
check the clarity of the questions and the flow and layout of the survey. 

On the basis of the pretests, we made revisions to both surveys. We 
contacted respondents to clarify any questions or responses where 
appropriate. Further, using a web-based survey and allowing state and 
district officials to enter their responses into an electronic instrument 
created an automatic record for each state and district and eliminated the 
errors associated with a manual data entry process. In addition, the 
programs used to analyze the survey data were independently verified to 
ensure the accuracy of this work. 


Expsrt Pansl To obtain information on lessons learned from RTT that could inform 

future education reform efforts, we convened a group of knowledgeable 
individuals for an expert panel. In identifying the experts, we compiled a 
preliminary list of 15 individuals with research or professional experience 
related to RTT reforms, state and district capacity, federal grant making, 
and state or federal education policy. These experts represented the 
following entities: state educational agencies, school districts, education 
associations, academia, and education think tanks. They also included a 
former Education official and a representative from Education’s Office of 
Inspector General. We identified a state educational agency official based 
on participation in RTT and the state’s proximity to Washington, D.C. 
where the panel was convened. 

To obtain a different local perspective, we selected a school district 
official from a different state. In addition, we selected the school district 
based on proximity to Washington, D.C. and the extent to which the 
district had completed questions in our district survey. An external expert 
who conducted extensive research on K-12 education and federal policy 
vetted our initial list of panelists. We used feedback from this expert, 
along with biographical information about the experts, to determine which 
experts would be invited to participate. 
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The resulting 10 experts participated in a 1-day panel focused on capacity 
challenges and their implications for RTT reforms and future competitive 
grants (see appendix V for list of participants). Each panelist completed a 
questionnaire to document any conflicts of interest. This information was 
not used to determine the qualification of the expert for the panel, but to 
ensure that we were aware of circumstances that could be viewed by 
others as affecting the expert’s point of view on these topics. We 
developed discussion topics and questions for the panelists based on 
information gathered from the surveys, interviews, and academic 
literature. A contractor recorded the panel and transcribed the discussion. 
We performed a content analysis of the transcript of the panel discussion 
to develop common themes among the experts on lessons learned from 
RTT that could help sustain reform efforts, inform the design or 
implementation of future education competitive grants, and inform future 
education reform efforts. We tallied responses for each panelist who 
commented on those themes. This analysis was independently verified to 
ensure the accuracy of this work. 


Review of Laws, 
Regulations, and 
Guidance and Interviews 
with Officials 


For all three objectives, we reviewed relevant federal laws, regulations, 
and guidance — including federal internal control standards and 
Education’s Handbook for the Discretionary Grant Process — and 
interviewed federal, state, and district officials and other experts regarding 
capacity to implement and sustain RTT reforms. We reviewed RTT 
applications to identify commitments states made to build capacity to 
implement RTT initiatives. To identify actions taken to build capacity, we 
compared the states’ commitments to information provided in their 
progress reports for school year 2012-2013. We also reviewed 
information on Education’s efforts to assist states with building capacity, 
such as guidance, technical assistance, webinars, and other information 
on the RTT website. We interviewed federal officials from the 
Implementation and Support Unit in Education’s Office of the Deputy 
Secretary and staff from the newly established Office of State Support. In 
addition, we conducted interviews with a variety of interested parties, 
such as educational organizations, researchers, and university 
professors. For example, we met with representatives from the American 
Association of School Administrators, the Council of Chief State School 
Officers, and the Center on Reinventing Public Education, among others. 
We also conducted follow-up interviews with officials in four state 
educational agencies and three districts to obtain more detailed 
information and illustrative examples. We selected these state and district 
officials based on their responses to our surveys and representation 
across award phase. 
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We conducted this performance audit from November 201 3 to Aprii 201 5 
in accordance with generaiiy accepted government auditing standards. 
Those standards require that we pian and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonabie basis for our 
findings and conciusions based on our audit objectives. We beiieve that 
the evidence obtained provides a reasonabie basis for our findings and 
conciusions based on our audit objectives. 
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RTT State 

Phase 

Award Amount 

Delaware 

1 

$119,122,128 

Tennessee 

1 

$500,741,220 

District of Columbia 

2 

$74,998,962 

Florida 

2 

$700,000,000 

Georgia 

2 

$399,952,650 

Hawaii 

2 

$74,934,761 

Maryland 

2 

$249,999,182 

Massachusetts 

2 

$250,000,000 

New York 

2 

$696,646,000 

North Carolina 

2 

$399,465,769 

Ohio 

2 

$400,000,000 

Rhode Island 

2 

$75,000,000 

Arizona 

3 

$25,080,554 

Colorado 

3 

$17,946,236 

Illinois 

3 

$42,818,707 

Kentucky 

3 

$17,037,544 

Louisiana 

3 

$17,442,972 

New Jersey 

3 

$37,847,648 

Pennsylvania 

3 

$41,326,299 

Total RTT Funding 


$4,140,360,632 


Source: U.S. Department of Education. | GAO-15-295 
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Category 

Criteria 

A. State Success Factors 

(A)(1) Articulating state’s education reform agenda and local educational 
agencies’ participation in it 


(A)(2) Building strong statewide capacity to implement, scale up, and sustain 
proposed plans 


(A)(3) Demonstrating significant progress in raising achievement and closing 
gaps 

B. Standards and Assessments 

(B)(1) Developing and adopting common standards 

(B)(2) Developing and implementing common, high-quality assessments 


(B)(3) Supporting the transition to enhanced standards and high-quality 
assessments 

C. Data Systems to Support Instruction 

(C)(1) Fully implementing a statewide longitudinal data system 
(C)(2) Accessing and using state data 
(C)(3) Using data to improve instruction 

D. Great Teachers and Leaders 

(D)(1) Providing high-quality pathways for aspiring teachers and principals 
(D)(2) Improving teacher and principal effectiveness based on performance 
(D)(3) Ensuring equitable distribution of effective teachers and principals 

(D)(4) Improving the effectiveness of teacher and principal preparation 
programs 

(D)(5) Providing effective support to teachers and principals 

E. Turning Around the Lowest-Achieving Schools 

(E)(1) Intervening in the lowest-achieving schools and local educational 
agencies 

(E)(2) Turning around the lowest-achieving schools 


F. General Criteria (F)(1) Making education funding a priority 


(F)(2) Ensuring successful conditions for high-performing charter schools and 
other innovative schools 

(F)(3) Demonstrating other significant reform conditions 


Source: 74 Fed. Reg. 59,688, 59,801-59,804 (Nov. 18, 2009). | GAO-15-295 

Note: In addition to these criteria. Education gave states the option to include other proposals in their 
plans, such as proposals to prepare more students for advanced study and careers in science, 
technology, engineering, and mathematics fields and proposals for states to work together to develop 
joint longitudinal data systems. 
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STATES (Percent and Number) 

DISTRICTS (Estimated Percentage) 

Standards and Assessments 

Organizational 

11% (2) 

23% 

Human Capital 

17% (3) 

31% 

Financial 

11% (2) 

34% 

Stakeholder 

39% (7) 

20% 

Data Systems 

Organizational 

24% (4) 

24% 

Human Capital 

18% (3) 

33% 

Financial 

24% (4) 

37% 

Stakeholder 

24% (4) 

20% 

Effective Teachers and Leaders 

Organizational 

22% (4) 

14% 

Human Capital 

11% (2) 

22% 

Financial 

22%(4) 

33% 

Stakeholder 

44% (8) 

16% 

School Turnaround 

Organizational 

40% (6) 

29% 

Human Capital 

27% (4) 

39% 

Financial 

33% (5) 

38% 

Stakeholder 

33% (5) 

28% 


Source: GAO survey of RTT states and currently participating districts. | GAO-1 5-295 


Note: For the districts, under the School Turnaround reform area, the percentage estimates for 
Human Capital and Financial capacity have 95 percent confidence intervals of +/- 7 percentage 
points. All other district estimates have confidence intervals within +/- 6 percentage points of the 
estimate itself. 
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1. Patricia E. Braxton 

Director of Curriculum and Instruction 
Woodstown-Pilesgrove Regional School District 

2. Keith Cummins 

Deputy Director, State and Local Advisory and Assistance Team 
Office of Inspector General, U.S. Department of Education 

3. Mary L. Gable 

Assistant State Superintendent, Division of Academic Policy and 
Innovation 

Maryland State Department of Education 

4. John Hill 
Executive Director 

National Rural Education Association 

5. Paul L. Posner 

Director of the Public Administration Program 
George Mason University 

6. Patrick McGuinn 

Associate Professor of Political Science and Education 
Drew University 

7. Penelope Thornton Talley 

Deputy State Superintendent for School Effectiveness & Chief 
Performance Officer 

Maryland State Department of Education 

8. Jeff Simering 

Director of Legislative Services 
Council of the Great City Schools 

9. Joanne Weiss 
President 
Weiss Associates 

10. Tiffany R. Winters 
Partner 

Brustein & Manasevit, PLLC 
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United States Department of Education 

Office of Elementary and Secondary Education 
Washington, D.C. 20202 

The Assistant Secretary 


March 30, 2015 


Jacqueline M. Nowicki, Director 
Education, Workforce, and 
Income Security Issues 
U.S. Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Ms. Nowicki: 

I am writing in response to the Government Accountability Office (GAO) draft report, 
“Race to the Top: Education Could Better Support Grantees and Help Them Address 
Capacity Challenges” (GAO- 15-295). We appreciate the time that your office devoted to 
this study and, in particular, the efforts that were made to de.scribe the support the 
Department has provided to Race to the Top grantees. 

As the GAO draft report notes. Race to the Top grantees have found the support provided 
by Department staff and our technical assistance contractor, the Reform Support Network 
(RSN), to be helpful. As part of the Department’s commitment to supporting states as they 
implement ambitious reform agendas, the Department established the Implementation and 
Support Unit (ISU) to administer, among others, the Race to the Top program. 

Specifically, the ISU and the RSN provided differentiated support based on individual state 
needs, and helped states work with each other and with experts to achieve and sustain 
educational reforms that improve student outcomes. 

At the end of 2014, the Department created the Office of State Support (OSS) to continue 
to provide support to States across programs as they implement comprehensive reforms. 
The OSS administers programs previously administered by the ISU, as well as a number of 
other programs, including those under Titles I, IIA, and II of the Elementary and 
Secondary Education Act, With a new state-centered strategy, the OSS aims to build 
greater .state capacity to drive implementation of comprehensive preK-12 education 
reforms across all Department programs administered by the OSS that will help schools 
strengthen their instructional systems and improve student outcomes. 

The report has offered recommendations, and we provide our response to these 
recommendations below. 

www.ed.gov 

400 MARYLAND AVE.. SW. WASHINGTON. DC 20202 

77ie Department of Education s mission is to promote student achie\'ement and preparation for global competitiveness hv 
fostering educational excellence and ensuring equal access. 
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Recommendation; To help ensure that states are better able to sustain RTT reforms and 
that Education can effectively support other grant programs managed by the Office of 
State Support, we recommend that the Secretary of Education direct the Office of State 
Support to incorporate into its monitoring and technical assistance plan or other policies 
and procedures the types of support that would be useful to RTT grantees. These could 
include: 

o providing individualized technical assistance to states, .such as that 
currently provided by Education program offtcials; 

o facilitating communities of practice to promote opportunities for 
collaboration across states; 

o providing professional development (or training) throughout the grant 
period, as opposed to only during the early stages of the grant; 

o making training more easily accessible by conducting training locally, 
when po.ssible; and 

o to the extent permissible in the context of federal and .state requirements 
and restrictions, exploring the possibility of assisting states in identifying 
skilled contractors to help implement reform efforts. 

Response: The Department shares GAO’s interest in supporting states as they sustain 
RTT reforms, as well as supporting other grant programs through performance 
management processes and technical assistance and notes that most of these 
recommendations, where possible, are already implemented. The OSS’s vision is to 
provide coordinated policy development, performance management, technical assistance, 
and data analysis services through a state support team structure that will deepen 
partnerships with states and more effectively support their implementation of key reforms 
and provide individualized support. 

Specifically, the OSS is developing performance management processes to build on the 
Department’s best practices to support and monitor grantees and align with state 
implementation efforts, including but extending beyond mere monitoring of compliance 
with statute, regulations, and policies to focus on quality of implementation across 
programs covered by the OSS. A consolidated performance management system will 
combine the OSS’s programs and identify methods to better provide grantees with 
continuous feedback on both the progress and quality of implementation, identify timely 
support and help grantees identify and address issues that result in more effective and 
efficient program implementation. In addition, the OSS will work to continue and enhance 
the technical assistance it provides to grantees including by ensuring effective support 
throughout grant periods. The OSS is developing plans to ensure timely, high-quality, and 
differentiated technical assistance, either through program offices, grants, or contracts to 
grantees individually and collectively, and plans to continue or replicate successful efforts, 
like the communities of practice. The OSS will Identify and disseminate best practices and 
lessons learned to build capacity, improve the quality and rigor of implementation, and 
ensure alignment and integration with the broader education agenda. We will conduct 
training locally when it is reasonable and efficient to do, for example when funding allows 
for travel. 
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Finally, we appreciate that the recommendation acknowledges that there are federal and 
state requirements and restrictions that limit the agency’s ability to assist states in 
identifying skilled contractors both because of the requirements of state procurement laws 
and policies, as well as limitations on Department employees from endorsing or 
sanctioning private entities. As you note in the report, we can and will continue to provide 
support to our grantees by helping them to develop objective criteria, analysis, or research 
as to the qualifications that skilled contractors would need in order to serve the purposes of 
the grant. We can also provide resource lists using objective criteria to that end. 

Recommendation.’ To help states address capacity challenges as they sustain 
comprehensive education reforms similar to RTT, we recommend that the Secretary of 
Education direct the Office of State Support to take steps, such as: 

o identifying ways to help states target assistance to rural districts; 
o finalizing and disseminating guidance to be included in Education "s toolkit 
to help states leverage federal formula grants to sustain education reforms; 
ami 

o clarifying and improving understanding of how funding flexibilities in 
existing formula grants could be used to help states and the audit 
community address impediments to using formula grants in different ways. 

Response: The Department will continue to support states by providing technical 
assistance and guidance regarding how to best use and leverage federal funds. As noted in 
GAO’S report, the Department has provided guidance to states regarding flexibility in 
using federal funds to meet local needs. Some of this guidance has been in the form of 
Dear Colleague letters.' In addition, the Department is working with the Reform Support 
Network to develop a toolkit for states and districts on ways to leverage federal grants to 
sustain educational reforms, to be relea,sed in 2015. Additionally, the OSS will provide 
coordinated policy development, performance management, technical assistance, and data 
analysis services through a state support team .structure that will support states’ 
implementation of key reforms across covered ESEA programs including by leveraging 
existing grant funds when allowable. 

The Department agrees that it is important to identify ways to help states target assistance 
to rural districts, and we have taken actions to do so. We do not believe the draft report 
acknowledges the actions that the Department has taken to date to support Race to the Top 
grantees in this way. We have provided a detailed list of these supports in the attachment 
to this response. 

We appreciate the opportunity to review the draft response and comment on the 
recommendations. 1 am also enclosing a document with technical comments. 


Sincerely, 



Deborah Delisle 
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' The Dear Colleague letters include guidance on the following topics: Maximizing Flexibility in the 
Administration of Federal Grants, httn://\vww2.ed.gov/Drograms teacheruual/flexibilitv-swpQQ 1 3 1 3.odf : 
Federal Programs and Support for School Counselors, 

http:'/wv\ vv2.cd.gov poiicv/elsec'guid/'secletter/l40630- !itml; and Leveraging ESEA and IDEA funds for 
Digital Teaching and Learning, http:/-tech.ed.gov/wn-contenl/uploads/20l4 l l/ Tech-Federa l -Funds-F inal- 
V2.pdf . 


Page 54 


GAO-15-295 Race to the Top Capacity 


Appendix VII: GAO Contact and Staff 
Acknowledgments 


GAO Contact 

Jacqueline M. Nowicki, (617) 788-0580 or nowickij@gao.gov 

Staff 

Acknowledgments 

In addition to the contact named above, Elizabeth Morrison (Assistant 
Director), Jamila Jones Kennedy (Analyst-in-Charge), Sheranda 
Campbell, Kathryn O’Dea Lamas, Amanda Parker, and Stacy Spence 
made significant contributions to this report. Assistance, expertise, and 
guidance were provided by David Chrisinger, Nancy Donovan, Alexander 
Galuten, Catherine Hurley, Jill Lacey, Jean McSween, Mark Ramage, 
Walter Vance, and Mimi Nguyen. 


( 131287 ) 


Page 55 


GAO-15-295 Race to the Top Capacity 


GAO’S Mission 

The Government Accountability Office, the audit, evaluation, and 
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